Combining Unsupervised Variable Selection with Dimensionality Reduction

This paper bridges the gap between variable selection methods (e.g Pearson coefficients, KS test) and dimensionality reduction algorithms (e.g PCA, LDA). Variable selection algorithms encounter difficulties dealing with highly correlated data, as many features are similar in quality. Dimensionality reduction algorithms tend to combine all variables, and are not able to select significant variables out of a set of features. Our approach combines both methodologies by applying variable selection followed by dimensionality reduction. The key point is to optimize the same utility function in both stages. The resulting algorithm is able to benefit from complex features as variable selection algorithms do, and at the same time enjoy the benefits of dimensionality reduction.